ci: Use vmactions for checks on other platforms
#2133
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2133 +/- ##
==========================================
- Coverage 93.33% 93.32% -0.01%
==========================================
Files 114 114
Lines 36887 36889 +2
Branches 36887 36889 +2
==========================================
Hits 34428 34428
- Misses 1679 1681 +2
Partials 780 780 ☔ View full report in Codecov by Sentry. |
Failed Interop TestsQUIC Interop Runner, client vs. server, differences relative to 6013bde. neqo-latest as client
neqo-latest as server
All resultsSucceeded Interop TestsQUIC Interop Runner, client vs. server neqo-latest as client
neqo-latest as server
Unsupported Interop TestsQUIC Interop Runner, client vs. server neqo-latest as client
neqo-latest as server
|
Benchmark resultsPerformance differences relative to 6013bde. decode 4096 bytes, mask ff: No change in performance detected. time: [11.137 µs 11.153 µs 11.179 µs]
change: [-0.3835% -0.1140% +0.1579%] (p = 0.42 > 0.05)
decode 1048576 bytes, mask ff: No change in performance detected. time: [3.0167 ms 3.0259 ms 3.0365 ms]
change: [-0.5709% -0.0869% +0.3625%] (p = 0.73 > 0.05)
decode 4096 bytes, mask 7f: No change in performance detected. time: [19.494 µs 19.543 µs 19.605 µs]
change: [-1.1043% -0.4302% +0.0660%] (p = 0.17 > 0.05)
decode 1048576 bytes, mask 7f: No change in performance detected. time: [5.1634 ms 5.1876 ms 5.2241 ms]
change: [-0.3159% +0.2212% +0.9739%] (p = 0.55 > 0.05)
decode 4096 bytes, mask 3f: No change in performance detected. time: [5.5266 µs 5.5502 µs 5.5791 µs]
change: [-0.8138% +0.8606% +3.5885%] (p = 0.63 > 0.05)
decode 1048576 bytes, mask 3f: No change in performance detected. time: [1.7598 ms 1.7652 ms 1.7720 ms]
change: [-0.4735% -0.0003% +0.4711%] (p = 0.99 > 0.05)
coalesce_acked_from_zero 1+1 entries: No change in performance detected. time: [98.783 ns 99.098 ns 99.422 ns]
change: [-0.3303% +0.0867% +0.4848%] (p = 0.68 > 0.05)
coalesce_acked_from_zero 3+1 entries: No change in performance detected. time: [116.59 ns 116.95 ns 117.34 ns]
change: [-0.8583% +0.0112% +0.7011%] (p = 0.98 > 0.05)
coalesce_acked_from_zero 10+1 entries: No change in performance detected. time: [116.30 ns 116.73 ns 117.25 ns]
change: [-0.4579% +0.2334% +0.8587%] (p = 0.51 > 0.05)
coalesce_acked_from_zero 1000+1 entries: No change in performance detected. time: [97.800 ns 97.954 ns 98.135 ns]
change: [-1.0089% +0.2774% +1.5403%] (p = 0.69 > 0.05)
RxStreamOrderer::inbound_frame(): No change in performance detected. time: [111.09 ms 111.22 ms 111.43 ms]
change: [-0.1373% +0.1060% +0.3182%] (p = 0.45 > 0.05)
SentPackets::take_ranges: No change in performance detected. time: [5.4235 µs 5.5244 µs 5.6413 µs]
change: [-1.6863% +0.4755% +2.8436%] (p = 0.69 > 0.05)
transfer/pacing-false/varying-seeds: Change within noise threshold. time: [42.144 ms 42.224 ms 42.306 ms]
change: [+0.9208% +1.2048% +1.4981%] (p = 0.00 < 0.05)
transfer/pacing-true/varying-seeds: Change within noise threshold. time: [42.427 ms 42.500 ms 42.574 ms]
change: [+0.9057% +1.1443% +1.3898%] (p = 0.00 < 0.05)
transfer/pacing-false/same-seed: Change within noise threshold. time: [42.178 ms 42.239 ms 42.301 ms]
change: [+0.7686% +1.0066% +1.2432%] (p = 0.00 < 0.05)
transfer/pacing-true/same-seed: Change within noise threshold. time: [42.715 ms 42.784 ms 42.860 ms]
change: [+1.3737% +1.6203% +1.8716%] (p = 0.00 < 0.05)
1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: No change in performance detected. time: [877.23 ms 887.16 ms 897.33 ms]
thrpt: [111.44 MiB/s 112.72 MiB/s 113.99 MiB/s]
change:
time: [-1.5464% +0.0297% +1.5359%] (p = 0.97 > 0.05)
thrpt: [-1.5126% -0.0297% +1.5707%]
1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: No change in performance detected. time: [300.72 ms 303.02 ms 305.37 ms]
thrpt: [32.748 Kelem/s 33.002 Kelem/s 33.253 Kelem/s]
change:
time: [-0.8400% +0.2123% +1.2972%] (p = 0.69 > 0.05)
thrpt: [-1.2806% -0.2119% +0.8472%]
1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: No change in performance detected. time: [34.375 ms 34.574 ms 34.792 ms]
thrpt: [28.742 elem/s 28.924 elem/s 29.091 elem/s]
change:
time: [-0.0851% +0.7820% +1.6247%] (p = 0.07 > 0.05)
thrpt: [-1.5988% -0.7759% +0.0852%]
1-conn/1-100mb-resp/mtu-1504 (aka. Upload)/client: Change within noise threshold. time: [1.6073 s 1.6220 s 1.6367 s]
thrpt: [61.100 MiB/s 61.654 MiB/s 62.216 MiB/s]
change:
time: [-3.6381% -2.2491% -0.8102%] (p = 0.00 < 0.05)
thrpt: [+0.8168% +2.3008% +3.7755%]
Client/server transfer resultsTransfer of 33554432 bytes over loopback.
|
larseggert
requested review from
KershawChang,
martinthomson and
mxinden
as code owners
|
@mxinden is the test failure legit? https://github.com/mozilla/neqo/actions/runs/12704862858/job/35414911552?pr=2133#step:5:3588 |
|
Thanks for the ping @larseggert. Failure should be fixed with #2339. |
|
FreeBSD now works, OpenBSD and Solaris have too old NSS packages. On NetBSD, |
There was a problem hiding this comment.
Overall looks good to me. I am fine merging early here to make sure we don't regress on the working platforms in the meantime.
I assume there is no easy way to reduce some of the repetition?
On a more general note, do I understand correctly that we are supporting these platforms on a best effort basis only?
We could move the scripts out of the
Totally. This is mostly to see if we can easily get things working for them. |
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Because building NSS and NSPR from source on these tier-2/3 platforms is a pain, the workflow relies on the packaged system NSS version. If that is too old for the current neqo code, checks for that OS are skipped.